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(54) POSITION DETECTOR 

(57)Abstract: 

PROBLEM TO BE SOLVED: To provide a position 
detector for a vehicle which implements correction of 
excluding an influence of pitching of the vehicle with a 
simple constitution to accurately detect a position. 
SOLUTION: Plural objects are extracted from a picture 
obtained by a camera, and an average value of extents 
of displacement of the positions in a (y) direction 
(perpendicular direction) of the plural objects is 
calculated as a pitching flow Ayp (S41). When the 
pitching flow Ayp is larger than a prescribed threshold A 
yTH, (y) coordinates of individual objects are corrected 
by the pitching flow Ayp (S42 and S43). 
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* NOTICES * 

JPO and INPiT are not responsible lor any 
danages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the location detection equipment which detects 
the location of an object based on the image obtained by the image pick-up equipment carried in 
the car. 
[0002] 

[Description of the Prior Art] Image pick-up equipment is carried in a car, the location of an 
object is detected from the image obtained by the image pick-up equipment, and the 
circumference supervisory equipment which judges the possibility of the collision with the object 
and car concerned is proposed conventionally. With such equipment, since it carries in the car 
which moves image pick-up equipment, an optical flow (henceforth a "pitching flow") occurs in 
the image obtained by the image pick-up equipment fixed to the car by pitching of a car. If it 
does not perform processing except the effect of a pitching flow in performing location detection 
of an object from the image obtained by image pick-up equipment, since this pitching flow is not 
what is generated by migration (or transit of the car concerned) of an object, it has the problem 
that a gap of a detection location becomes large. 
[0003] 

[Probiem(s) to be Solved by the Invention] Therefore, although it is possible to amend a 
detection location based on the car height which forms the car height sensor which detects the 
height (car height) from the road surface of a car, and is detected by this sensor, a car height 
sensor is needed in that case, and the structure of equipment is complicated. 
[0004] This invention is made paying attention to this point, amendment except the effect of 
pitching of a car is realized with an easy configuration, and it aims at offering the location 
detection equipment for cars which can perform exact location detection. 
[00053 

[Means for Solving the Problem] In order to attain the above-mentioned purpose invention 
according to claim 1 in the location detection equipment which detects the location in the real 
space system of coordinates of an object based on the image obtained by the image pick-up 
means carried in the car the image obtained by said image pick-up means to two or more 
objects — extracting — the location of the height direction of two or more of these objects — a 
variation rate — with an amount calculation means of amendments to compute the amount of 
pitching amendments according to an amount It considers as the having-amendment means to 
amend location on image of two or more of said objects according to said amount of pitching 
amendments description. 

[0006] According to this configuration, two or more objects are extracted from the image 
obtained by the image pick-up means, the location of the height direction of two or more of 
these objects — a variation rate, since the amount of pitching amendments is computed 
according to an amount and the location on the image of two or more of said objects is amended 
according to said amount of pitching amendments pitching of the car concerned — the location 
of the height direction in an image — even when a variation rate occurs, except for the effect, 
exact location detection can be performed with an easy configuration, without using a car height 
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sensor etc, 

[0007] When it is desirable to compute the average value of said amount of location 
displacement about said two or more objects, and to make this average value into said amount of 
pitching amendments as for said amount calculation means of amendments and said average 
value of said amendment means is larger than a predetermined threshold, it is desirable to 
perform amendment by said amount of pitching amendments, said location about two or more 
objects — a variation rate — the location on the image according to factors other than pitching 
(transit of the car concerned, or migration on the real space of an object) when the average 
value of an amount is large — a variation rate — the location on the image according [ an 
amount ] to pitching — a variation rate — since it is thought compared with an amount that it is 
very small, the effective amount of pitching amendments can be obtained with an easy 
configuration by this configuration. 
[0008] 

[Embodiment of the Invention} With, the gestalt of operation of the Shimomoto ** is explained 
with reference to a drawing. Draw ing 1 is drawing containing the location detection equipment 
concerning 1 operation gestalt of this invention showing the configuration of the circumference 
supervisory equipment of a car. This equipment Two infrared cameras 1R and 1L which can 
detect far infrared rays, and the yaw rate sensor 5 which detects the yaw rate of the car 
concerned, The speed sensor 6 which detects the travel speed (vehicle speed) VCAR of the car 
concerned, and the brake sensor 7 for detecting the control input of a brake, The image- 
processing unit 2 which detects the object of the animal ahead of a car etc. based on the image 
data obtained with these cameras 1R and 1L, and emits an alarm when the possibility of a 
collision is high, It has the loudspeaker 3 for emitting an alarm with voice, and HUD (henceforth 
"HUD") 4 for making an operator recognize an object with the high possibility of a collision, while 
displaying the image obtained with Cameras 1R or 1L. 

[0009] Cameras 1R and 1L are being fixed so that it may be arranged to the medial axis of the 
longitudinal direction of a car 10 in the almost object location at the anterior part of a car 10 as 
shown in drawing 2 , and the optical axis of two cameras 1R and 1L may become parallel 
mutually and the height from both road surface may become equal. Infrared cameras 1R and 1L 
have the property that the output-signal level becomes high (brightness increases), so that the 
temperature of an object is high. 

[0010] The A/0 -conversion circuit where the image-processing unit 2 changes an input analog 
signal into a digital signal, The image memory which memorizes the digitized picture signal, CPU 
which performs various data processing (Central Processing Unit), RAM used in order that CPU 
may memorize the data in the middle of an operation (Random Access Memory), ROM which 
memorizes the program which CPU performs, a table, a map, etc. (Read Only Memory), It has the 
output circuit which outputs the driving signal of a loudspeaker 3, the status signal of HUD4, etc., 
and the output signal of Cameras 1R and 1L and sensors 5-7 is changed into a digital signal, and 
it is constituted so that it may be inputted Into CPU. As shown in d rawing 2 , HUD4 is formed so 
that screen 4a may be displayed on the front window s of a car's 10 front location of an 
operator. 

[00113 Drawing 3 is a flow chart which shows the procedure of location detection processing of 
the object in the image-processing unit 2, carries out A/D conversion of the output signal of 
Cameras 1R and 1L first, and stores it in an image memory (steps S1 1, S12, and S13). The image 
data stored in an image memory is data of a gray-scale image including brightness information. 
For each, drawing 5 (a) and (b) are a gray-scale image (a right image is obtained by camera 1R) 
obtained wTth Cameras 1R and 1L. a left image obtains by camera 1L — having — the field 
which is drawing for explaining and attached hatching It is the field of middle gradation (gray) and 
the field enclosed with a thick continuous line is a field (henceforth a "high brightness field ) of 
the object with which an intensity level is highly displayed as white on a screen (at elevated 
temperature). By the right image and the left image, since the horizontal position on the screen 
of the same object shifts and is displayed, the distance to that object is computable with this 
gap (parallax). 

[001 2] At step $14 of drawing 3 , processing which uses a right image as a criteria image, sets to 
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"1" (white) binary-ization of the picture signal, i.e., a field brighter than the brightness threshold 
ITH determined experimentally, and sets a dark field to "0" (black) is performed. The image 
which made the image of drawing 5 (a) binary to dra wing 6 is shown. The field which attached 
hatching is black and this drawing shows that the "high brightness field surrounded as the thick 
continuous line is white. 

[0013] At continuing step SI 5, processing which changes into run length data the image data 
made binary is performed. Drawing 7 (a) is drawing for explaining this, and shows the field which 
became white by binary-ization as Rhine LH8 on pixel level in this drawing. For explanation, it 
estranges and Rhine L1-U is shown, although each has width of face of 1 pixel in the direction 
of y and it has stood in a line in the direction of y without the clearance in fact. Moreover, Rhine 
L1-L8 has die length of 2 pixels, 2 pixels, 3 pixels, 8 pixels, 7 pixels, 8 pixeis, 8 pixels, and 8 
pixels in the x directions, respectively. Run length data show Rhine L1--L8 by the coordinate of 
the start point (point at the left end of each Rhine) of each Rhine, and the die length (pixel 
number) from a start point to an ending point (point at the right end of each Rhine), for example, 
Rhine L3 — (x3, y5) — and (x4, y5) (x5, y5) — since it consists of 3 pixels — as run length data 
— ( — it becomes x3, y5, and 3). 

[0014] At steps S16 and S17, processing which extracts an object is performed by carrying out 
labeling of an object, as shown in drawing 7 (b). That is, it considers that Rhine L1-L3 with the 
part which laps in the direction of y among Rhine L1-L8 formed into run length data is one 
object 1, and considers that Rhine L4-L8 is one object 2, and the object labels 1 and 2 are 
added to run length data. By this processing, the high brightness field shown in drawing 6 will be 
grasped as objects 1-4, respectively. 

[0015] At step S18, as shown in drawing 7 (c), the aspect ratio ASPECT of the circumscribed 
quadrangle shown with the area center of gravity G, the area S, and the broken line of the 
extracted object image is computed, the ratio of Dy and Dx which compute area S by integrating 
the die length of run length data about the same object, compute the coordinate of the area 
center of gravity G as a y-coordinate of the line equally divided into two along the x-coordinate 
of a line which divides area S into two equally along the direction of y, and x directions, and show 
an aspect ratio APECT to dra wing 7 (c) — it computes as Dy/Dx. In addition, the center-of- 
gravity location of a circumscribed quadrangle may be substituted for the location of a center of 
gravity G. 

[0016] At step SI 9, the same object is recognized for every trace during time of day of an 
object, l.e„ a sampling period. Time of day discretized with the sampling period in the time of day 
t as an analog quantity is set to k, and the identity judging with the objects 3 and 4 extracted at 
time of day (k+1) when objects 1 and 2 were extracted at time of day k, as shown in drawing s 
(a), and objects 1 and 2 is performed. When fulfilling the following identity criteria 1-3, 
specifically, the trace during time of day is performed by judging with objects 1 and 2 and objects 
3 and 4 being the same, and changing objects 3 and 4 into a label called 1 and 2, respectively. 
[0017] 1) The center-of-gravity position coordinate on the image of the object i in time of day k 
(=1 2) When it considers as each (xi (k), yi (k)) and the center-of-gravity position coordinate on 
the image of the object j (=3 4) in time of day (k+1) is set to (xj (k+1), yj (k+D), |xj(k+1)-xi — it 
is (k) {^leltaxM yj(k+1 )-yi(k) |<deitayM. However, delta xM and delta yM are the allowed values of 
the amount of displacement on the image of x directions and the direction of y, respectively. In 
addition, allowed value deltayM of the direction of y also takes into consideration the amount of 
displacement on the image by pitching of a car 10, and is set up. 

[0018] 2) When you set area on the image of the object i in time of day k (= 1 2) to Si (k) and 
you set area on the image of the object j (=3 4) in time of day (k+1) to Sj (k+1), it is Sj (k+1) /Si 
(k) <j*#deitaS. However, deltaS is the allowed value of area change. 

[0019] 3) When you set the aspect ratio of the circumscribed quadrangle of the object i in time 
of day k (=1 2) to ASPECTi (k) and you set the aspect ratio of the circumscribed quadrangle of 
the object j (=3 4) in time of day (k+1) to ASPECTj (k+1), it is ASPECTj(k+1)/ASPECTi(k) 
<3 **deltaASP EC T. However, deitaASPECT is the allowed value of aspect ratio change. 
[0020] Since objects 1 and 3 will fulfill the above-mentioned identity criteria and objects 2 and 4 
will fulfill the above-mentioned identity criteria although, as for each object, the magnitude on an 
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image is [ the direction of this drawing (b) ] large if (b) is contrasted with cjrawing_8 (a), objects 3 
and 4 are recognized to be objects 1 and 2, respectively. Thus, the position coordinate (center of 
gravity) of each recognized object is stored in memory as time series location data, and is used 
for next data processing. At step S20 of drawin g 3 , when the optical flow on an image occurs by 
the pitching amendment processing shown in drawin g 4 , i.e., pitching of a car 10, processing 
which amends the location on the image of an object is performed. 

[0021] several objects with which dra wing 9 is extracted — the case of N=3 — an example — 
taking — the variation rate on the image of each object — the variation rate on the image of 
each object at the time of presupposing that there were not vectors VM1-VM3, the pitching 
vector **** which is the optical flow generated by pitching, and pitching — it is drawing showing 
the relation of vector VM1 p-VM3p. OB JOLDi and OBJNOWi show a location in this drawing the 
last location (location in front of 1 sample period) of Object i (=1,2, 3), and this time, 
respectively. „ 
[0022] At step S41 of drawin g 4 , amount of direction displacement of y (henceforth pitching 
flow") deltayp on the image generated by pitching is computed by the following type (1). 
[Equation 1] 

Ayp= __L|; (yiNOW-yiOLD) (1) 

yiNOW and yiOLD are the y-coordinate of the center of gravity of this object i 0=1 -N), and 
the y-coordinate of the center of gravity of the last (before 1 sample period) object i here, 
respectively, and N is the number of the objects currently extracted, the y-coordinate to which 
pitching flow deltayp of a formula (1) makes min the performance index E shown by the following 
formula (2) — a variation rate — it computes as amount deltay — having — the direction of y 
per unit time amount (this operation gestalt per 1 sample period) on the image of the object of N 
individual — a variation rate — it is equivalent to the average of an amount. The amount of 
direction displacement of y on the image resulting from transit of the self-car 10, or migration of 
an object (the variation rate of the direction of y of an object in case there is no pitching 
amount) The optical flow (pitching vector ****) generated by the very small thing and pitching 
compared with the amount of displacement generated by pitching Since it is not dependent on 
the location on an image and can approximate uniformly about all objects, this operation gestait 
defines pitching flow deltayp as deltay which makes the performance index E of a formula (2) 
min. 

[Equation 2] 

Er=: ^ (yiNOW— yiOLD+ Ay) (2) 

[0023] Subsequently, pitching amendment is performed by the absolute value of pitching flow 
deltayp distinguishing whether it is larger than predetermined threshold deltayTH (step S42), and 
judging with pitching of a car 10 being large, and adding pitching flow deltayp to y-coordinate 
yiNOW of Object i 0=1 -N) t when it is ileitayp I>deltayTH (step S43). In addition, predetermined 
threshold deltayTH is set as a suitable value by experiment. 

[0024] The variation rate (variation rate of the direction of y of an object in case there is no 
pitching) of the direction of y on the image which originates in migration of the self-car 10 or an 
object as described above Since it is very small compared with the variation rate generated by 
pitching, when the absolute value of pitching flow deitayp computed by the formula (1) exceeds 
predetermined threshold deltayTH It is regarded as the amount of displacement which generated^ 
all deltaypfe) by pitching, and a y-coordinate is amended. The location shown by OBJNOWip (2 i= 
h 3) of d rawing 9 is equivalent to the location after amendment of each object. 
[0025] On the other hand, when it is |deltayp[<=deltayTH at step S42, it judges with the ability of 
the effect of pitching to be disregarded, and progresses to step S44 immediately, and 
amendment of a y-coordinate is not performed. At step S44, this time value yiNOW of the y- 
coordinate of each object is made into a value yiOLD last time, and this processing is ended. 
Thereby, in the processing after 1 sample period, a value is used as a value last time this time in 
this processing. 
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[0026] Since according to processing of drawing 4 amount of displacement deltayp of the 
direction of y by pitching of the self-car 10, i.e., the height direction, is computed based on the 
image itself obtained by camera 1R and the y-coordinate of an object is amended by this, 
pitching amendment can be realized with an easy configuration and it becomes possible to 
detect more correctly the location of the" object in the real space system of coordinates 
explained below as a result. In addition, processing of steps S14-S20 of drawing^ explained 
above was made binary, is criteria-image (this operation-gestait righHmage )-attached, and is 

performed. „ t . .... ni 

[0027] It asks for the object image R2 in the left image corresponding to the object image Rl 
contained in a right image at step S31 of drawing 3 as shown in drawjngJO . The distance dR of 
the center line LCTR of the x directions of a right image and the center of gravity of the object 
image R 1 and the distance dL of the center line LCTR of a left image and the center of gravity 
of the object image R2 are computed, and the distance z from the self-car 10 to the object is 
computed with the application of these at a following ceremony (3). Since long time amount is 
required compared with processing of steps S19 and S20, step S31 is performed a period (for 
example, about 3 times [of the execution cycle of steps S11-S20 ] period) longer than steps 
S19 and S20. 

[Equation 3] 

bxf bxf 

Z " <dL+dR>Xp AdXp 

[0028] Here, as B is shown to the base length, i.e., d^ngJl , the focal distance of Lenses 12R 
and 12L and p of the distance (spacing of the optical axis of both cameras) of the horizontal 
direction (x directions) of the center position of image sensor 1 1R of camera 1R and the center 
position of image sensor 11 L of camera 1L and F are pixel spacing in image sensor 1 1 R and 11L, 
and deltad NR+dL) is the amount of parallax. Moreover, f is the ratio of a focal distance F and 

the pixel spacing p. , , , M , . 

[0029] At step S 21, the distance z computed by the coordinate (x y) and formula (3) in an image 
is applied to the following formula (4), and it changes into the coordinate (X, Y, Z) in real space 
system of coordinates. Here, by making the core of an image into a zero, it is horizontally 
determined as x and the system of coordinates in an image have determined it perpendicularly as 
y, as are shown in drawing 12 (a), and real space system-of-coordinates X-Y-Z makes Zero O 
the location (location fixed to the self-car 10) of the middle point of the installation location of 
Cameras 1R and 1L, defines it like illustration and shows in this drawing (b). 

<4> 

z 

f=F/p 

[0030] Here, (xc, yc) change the coordinate on a right image (x y) into the coordinate in the 
imagination image which made in agreement the installation location of camera 1R, and relative- 
position relation machine ****** with the real space zero O, the real space zero O and the core 
of an image. Moreover, f is the ratio of a focal distance F and the pixel spacing p. 
[0031] As mentioned above, with this operation gestalt, since pitching flow deltayp which shows 
a location gap of the direction of y resulting from pitching of the self-t:ar 10, i.e., the height 
direction, is computed from the image data obtained by camera 1R and the y~coordinate of an 
object was amended, exact location data can be obtained except for the effect of pitching of the 

S 6 lf"*"C3tf 10, 

[0032] Based on the positional information on the real space of the object computed by 
processing of drawing 3 , the image-processing unit 2 judges the possibility of the collision with 
the object and self-car 10, and when the possibility of a collision is high, it emits the alarm to an 
operator through a loudspeaker 3 and HUD4. 



Equation 4] 
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xcXz/f 
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[0033] With this operation gestalt, the image -processing unit 2 constitutes location detection 
equipment, steps S16-S 19 of drawing_3 and step S41 of drawin g_4 are equivalent to the amount 
calculation means of amendments, and, more specifically, steps S42 and S43 of drawing. 4 are 
equivalent to an amendment means. In addition, this invention is not restricted to the operation 
gestalt mentioned above, and various deformation is possible for it. for example, the variation 
rate of the y~coordinates of all the objects extracted with the operation gestalt mentioned above 
— the variation rate of some (however, it considers as at least two or more objects) y- 
coordinates of the object extracted although the average value of an amount was set to pitching 
flow deltayp — you may make it compute pitching flow deitayp from an amount 
[0034] Moreover, with this operation gestalt, although the infrared camera was used as an image 
pick-up means, as shown, for example in J P,9-226490,A, the television camera which can detect 
only the usual visible ray may be used. However, by using an infrared camera, extract processing 
of the car under transit, such as an animal, etc. can be simplified, and the arithmetic proficiency 
of an arithmetic unit can also realize a comparatively low thing. 

[0035] ..... 
[Effect of the Invention] As explained in full detail above, according to this invention, two or 
more objects are extracted from the image obtained by the image pick-up means, the location of 
the height direction of two or more of these objects — a variation rate, since the amount of 
pitching amendments is computed according to an amount and the location on the image of two 
or more of said objects is amended according to said amount of pitching amendments pitching of 
the car concerned — the location of the height direction in an image — even when a variation 
rate occurs, except for the effect, exact location detection can be performed with an easy 
configuration, without using a car height sensor etc. 
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* NOTICES * 

JPO and INPIT are not responsible for any 
dasages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.ln the drawings, any words are not translated. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 13 It is drawing containing the location detection equipment concerning 1 operation 
gestalt of this invention showing the configuration of the circumference supervisory equipment 
of a car. r 
[Drawing^]. It is drawing for explaining the installation location of an infrared camera. 
[Drawing 3] It is the flow chart which shows the procedure of processing of performing location 
detection which extracted the object. 

[Draw ing 4] It is the flow chart of pitching amendment processing of drawmgjf - 
[Drawing 51 In order to explain the gray-scale image obtained by the infrared camera, it is 
drawing attaching and showing hatching in the middle gradation section. 

[Drawing 6] In order to explain the image which made the gray-scale image binary, it is drawing in 
which giving hatching and showing a black field. 

[Drawi ng 7j it is drawing for explaining transform processing and labeling to run length data. 

[Dr awing 8 ] It is drawing for explaining the trace during time of day of an object. 

[Dr awing 9] It is drawing for explaining pitching amendment. 

[Drawing 10] It is drawing for explaining the calculation approach of parallax. 

[Drawing 11] It is drawing for explaining the technique of computing distance from parallax. 

[Drawing 12] It is drawing showing the system of coordinates in this operation gestalt. 

[Description of Notations] 

1R, 1L Infrared camera (image pick-up means) 

2 Image-Processing Unit (the Amount Calculation Means of Amendments, Amendment Means) 



[Translation done.] 
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* NOTICES * 

JPO and INPIT are not responsible for any 
dasages caused by the use of this translation. 

1. This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2. **** shows the word which can not be translated. 
3 Jn the drawings, any words are not translated. 



CLAIMS 



[Claim 1] In the location detection equipment which detects the location in the real space 
system of coordinates of an object based on the image obtained by the image pick-up means 
carried in the car the image obtained by said image pick-up means to two or more objects — 
extracting — the location of the height direction of two or more of these objects — a variation 
rate — with an amount calculation means of amendments to compute the amount of pitching 
amendments according to an amount Location detection equipment by which it is having- 
amendment means to amend location on image of two or more of said objects according to said 
amount of pitching amendments characterized. 
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* NOTICES * 

JPO and INP1T are not responsible for any 
damages caused by the use of this translation, 



t.This document has been translated by computer So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated, 
3Jn the drawings, any words are not translated. 



DRAWINGS 
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